(1) Field of the Invention
The present invention relates to a moving picture multiplexing apparatus which multiplexes coded audio data and video data to generate a seamlessly-reproducible multiplexed stream, an audio and video recording apparatus, and a moving picture multiplexing method.
(2) Description of the Related Art
Recent introduction of high-definition camcorders in increasing numbers has promoted development of Advanced Video Codec High Definition (AVCHD)-compliant camcorders. Such camcorders store in a storage medium a single audio and video stream including the start of recording through the end of recording, as well as an AVCHD-compliant navigation file. By repeating the above storage operations, audio and visual stream files and navigation files corresponding to the audio and visual stream files are stored in the storage media.
Here, the AVCHD-compliant navigation files include the following:                A clip information file in which information on audio and video streams is stored;        A play list file in which information on a reproduction order of the audio and video streams is stored;        An object file in which information on an reproduction order of the play lists is stored; and        An index file in which overall information on the storage medium is stored.        
In reproducing the audio and video streams in a reproduction order stored in the play list file, the AVCHD standard introduces a technique capable of sequentially and seamlessly reproducing the audio and video streams. Thus, the AVCHD-compliant reproducing devices are designed to seamlessly reproduce audio and video streams designated for seamless reproduction.
In order to seamlessly reproduce the audio and video streams sequentially, however, the following conditions shall be met at once:
<Conditions of Video Coding Information>
                The same resolution among the streams;        The same frame rate among the streams;        The same aspect ratio among the streams;        The same number of referable pictures used in motion compensation; and        Compliant with the virtual buffer model, including connection points (No overflow or underflow is developed).<Conditions of Audio Coding Information>        The same number of channels among the streams;        The same bit rate among the streams;        The same sampling frequency among the streams;        Compliant with the virtual buffer model of the multiplex system, including connection points (No overflow or underflow is developed).        The value of the start Presentation Time Stamp (PTS) of audio should be equal to or smaller than that of the video start PTS.        A time indicating the last PTS of the audio in addition to a time period for one audio frame should be equal to or greater than a time indicating the last PTS of the video in addition to a time period for one video frame.<Conditions of Stream Multiplexing>        
Conditions of multiplexing are shown in FIG. 1. FIG. 1 shows a relationship between a system clock and a PTS. Here, the PTS is assigned to a connecting part found when a preceding stream TS1 and a following stream TS2 are seamlessly reproduced. In the Description, the terms “system clock” and “PTS” are defined in the same meaning level as below. The “system clock”, typically used in multiplexing, means “a time at which the audio and video data is multiplexed”. The “PTS” means “a time at which video is displayed” or “a time at which audio is delivered”.
First, defined as follows are PTSs and system clocks of the preceding stream TS1 (hereinafter referred to as TS1) and the following stream TS2 (hereinafter referred to as TS2):                PTS1_1_end: the last PTS of a video of TS1; PTS2_2_start: the start PTS of the video of TS2;        Tpp: a time period interval for displaying the last picture included in the video of the TS1;        STC_delta=PTS1_1_end+Tpp−PTS2_2_start: a difference value used for converting the system clock on the TS1 into that on the TS2;        STC2=STC1−STC_delta: an equation converting the clock on TS1 into that on TS2;        STC1_1_end: a system clock value, on TS1, at which the end data of the last packet of TS1 is transmitted;        STC2_2_start: a system clock value, on TS2, at which the start data of the start packet of TS2 is transmitted; and        STC2_1_end=STC1_1_end−STC_delta: a value obtained by converting the system clock value at which the end data of the last packet of TS1 into a clock on TS2        
In order to reproduce TS1 and TS2 seamlessly, the condition described below shall be met in performing multiplexing.
<Conditions for a Successful Seamless Connection>
                STC2_2_start>STC2_1_end        
Following the last packet of TS1, the start packet of the TS2 is provided with the system clocks of the TS1 and TS2 unbroken. Then, the system clock, the PTS, and a Decoding Time Stamp (DTS), all of which are assigned to all packets included in TS2, are converted into the system clocks on TS1 in conformity with the system clocks found on the TS1 and having the start packet of TS2. Here, in performing the conversion, the PTSs and the DTSs of the video are to be matched with a predetermined frame rate. The above configuration makes possible seamlessly reproducing TS1 and TS2.
A conventional art has proposed a technique for generating a stream capable of the seamless reproduction (See Patent Reference 1: Japanese Unexpected Patent Application Publication No. 2009-4897). The technique involves generating a seamlessly reproducible stream as follows: upon ending storage of a preceding stream, the technique involves holding information on virtual buffer occupancy and multiplexing of following streams to be successively stored; and upon starting the storage of the following streams, the technique employs the held information to involve adjusting the amount of coded data of the following streams, and timing the multiplexing of the following streams.
The above conventional technique can ensure a seamless connection when reproducing streams in a storing order; however, the conventional technique cannot meet all the conditions for the seamless connection in the case where the streams are reproduced in an order different from the storing order.
FIG. 2 shows how the conventional technique develops problems when changing the reproduction order of audio and video streams stored by the conventional technique.
For example, suppose audio and video streams are obtained by a camcorder, and the streams are stored in a storage medium as shown in FIG. 2. Then, the audio and video streams in the storage medium are loaded on a personal computer (hereinafter referred to as PC), changed in reproduction order, and stored in another storage medium. Here, in some sequential streams, the above conditions for stream multiplexing cannot be met, resulting in possible failure of seamless reproduction of the streams.
In the case where Stream 2 shown in FIG. 2 and found between Streams 1 and 3 is deleted, Streams 1 and 3 might not be seamlessly reproduced as well.
In order to achieve a successful seamless connection after a change of a reproduction order of streams or deleting a stream, the following streams hive to be re-multiplexed so that each following stream can be seamlessly connected to a corresponding preceding stream in a reproduction order. This process causes an increase in processing load. As described above, the conventional technique might develop a possible failure of seamless reproduction when reproducing streams in an order different from the storing order.
The present invention is conceived in view of the above problems and has as an object to introduce a moving picture multiplexing apparatus capable of generating seamlessly-reproducible multiplexed streams even in the case where the streams are reproduced in an order different from an order in which the streams have been stored, an audio and video recording apparatus, and a moving picture multiplexing method.